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background: Nomograms are predictive tools that are widely used for estimating cancer prognosis. The aim of this study was 
to develop a nomogram for the prediction of overall sun/ival (OS) in patients diagnosed with cervical cancer. 
methods: Cervical cancer databases of two large institutions were analysed. Overall survival was defined as the clinical endpoint and 
OS probabilities were estimated using the Kaplan-Meier method. Based on the results of survival analyses and previous studies, 
relevant covariates were identified, a nomogram was constructed and validated using bootstrap cross-validation. Discrimination of the 
nomogram was quantified with the concordance probability. 

results: In total, 528 consecutive patients with invasive cervical cancer, who had all nomogram variables available, were identified. 
Mean 5-year OS rates for patients with International Federation of Gynecologists and Obstetricians (FIGO) stage lA, IB, II, III, and IV 
were 99.0%, 88.6%, 65.8%, 58.7%, and 41.5%, respectively. Seventy-six cancer-related deaths were observed during the follow-up 
period. FIGO stage, tumour size, age, histologic subtype, lymph node ratio, and parametrial involvement were selected as nomogram 
covariates. The prognostic performance of the model exceeded that of FIGO stage alone and the model's estimated optimism- 
corrected concordance probability was 0.723, indicating accurate prediction of OS. We present the prediction model as nomogram 
and provide a web-based risk calculator (http://www.ccc.ac.at/gcu). 

conclusion: Based on six easily available parameters, a novel statistical model to predict OS of patients diagnosed with cervical 

cancer was constructed and validated. The model was implemented in a nomogram and provides accurate prediction of individual 

patients' prognosis useful for patient counselling and deciding on follow-up strategies. 
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The annual incidence of invasive cervical cancer is 6.6/100 000 
women in Austria. Within the year 2009, 394 new cases of cervical 
cancer were diagnosed (Statistik Austria, 2011). Clinical stage 
based upon the International Federation of Gynecologists and 
Obstetricians (FIGO) system reflects the strongest prognostic 
parameter in patients with cervical cancer (Benedet et a/, 2000). 
The FIGO staging system is based upon clinical examination and 
the system's limitations are well appreciated (Hricak et al, 2005). 
A number of other clinico-pathological parameters were shown to 
have additional prognostic value in predicting patients' prognosis 
(Lagasse et al, 1980). Besides FIGO stage, the most notable 
prognostic factors are lymph node status and the number of 
positive lymph nodes, tumour size, parametrial involvement, 
histologic type, and patients' age (Delgado et al, 1990; Inoue and 
Morita, 1990; Takeda et al, 2002; Quinn et al, 2006). The prognostic 
relevance of other additional parameters such as lymph vascular 
space invasion (LVSI), histologic grade, and depth of cervical 
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stromal invasion is controversial (Creasman and Kohler, 2004; 
Kastritis et al, 2007). 

Prognostic nomograms have been recently introduced for 
numerous malignancies, including gynaecologic cancers, to obtain 
reliable prognostic information tailored to each individual patient 
(Chi et al, 2008; Abu-Rustum et al, 2010). Several nomograms were 
shown to compare favourably to traditional staging systems and 
their use has been suggested in addition to or even as an alternative 
to disease stage and other established prognosticators (lasonos et al, 
2008; Abu-Rustum et al, 2010). For patients with cervical cancer, a 
nomogram to predict overall survival (OS) through stages I-IV has 
not been published so far. The aim of this study was to develop and 
validate a nomogram for individual prediction of OS in patients 
diagnosed with invasive cervical cancer. 



PATIENTS AND METHODS 
Patients 

All consecutive patients diagnosed with cervical cancer at the 
University Hospitals Vienna and Innsbruck, Austria between 1996 
and 2009 were eligible for this study. Clinical data were extracted 
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from the institutions' prospectively maintained, electronic data- 
bases. Institutional review board approval was obtained before the 
study (IRB approval numbers: 246-2009 (Ethics Committee 
Medical University of Vienna) and UN4144 (Ethics Committee 
Medical University of Innsbruck)). 

Clinical management 

Patients were treated and followed according to the international 
guidelines (Committee on Practice Bulletins-Gynecology, 2002; 
Greer et al, 2008). Briefly, patients diagnosed at stage lAl disease 
were treated with conization or simple hysterectomy. If final 
pathology revealed the presence of LVSI, systematic pelvic and/or 
paraaortic lymphadenectomy was performed. Patients with stage 
IA2 were treated either by conization or simple hysterectomy plus 
systematic pelvic and/or paraaortic lymphadenectomy. Early stage 
disease FIGO IBl-IIA was managed by radical hysterectomy 
or trachelectomy plus systematic pelvic and/or paraaortic lym- 
phadenectomy. Patients with positive lymph nodes, parametrial 
involvement, and positive or close surgical margins were treated 
with adjuvant, postoperative radiotherapy - with or without 
concurrent chemotherapy. Cervical cancer FIGO IIB-IV was 
treated with primary radio-chemotherapy. In these patients, pelvic 
and/or paraaortic lymphadenectomy was performed before radio- 
chemotherapy to allow individualised planning of the radiation 
field. Extended field radiation was administered to patients with 
positive lymph nodes in the common iliac region (when paraaortic 
lymphadenectomy was not performed) or to patients with positive 
paraaortic lymph nodes (Potter et al, 2011). 

Statistical analysis 

Values are given as mean (standard deviation (s.d.)) when normally 
distributed or as median (interquartile range (IQR)) at the presence of 
skewed distribution. P- values of <0.05 were considered statistically 
significant. We used the statistical software R 2.12.2 for Windows 
(R Foundation for Statistical Computing, Vienna, Austria). 

Overall survival, defined as the time from diagnosis to date of 
cancer-related death or date of last follow-up for patients who 
were alive and censored, was defined as the clinical endpoint. 
Overall survival probabilities were estimated using the Kaplan- 
Meier method. The following predictors were selected for model 
development based on previous multivariable analyses (Polterauer 
et al, 2011): age (continuous variable), FIGO stage (IV vs III vs II vs 
IB), the presence of positive lymph nodes (positive vs negative), 
and at the presence of positive lymph nodes lymph node ratio 
(LNR; continuous variable), parametrial involvement (positive vs 
negative), and tumour size ( ^2 vs >2 cm). Lymph node ratio was 
defined as the ratio of positive and totally removed lymph nodes, 
as previously described (Polterauer et al, 2010). FIGO stages I A 
were excluded from predictive modelling, as only one event was 
observed in this group. The model was constructed by estimating a 
Cox regression model, maximising a likelihood function with a 
ridge (sum of squared regression coefficients) penalty term to 
prevent against over-fitting potentially caused by the small number 
of events in our study (76 events). This type of analysis can deal 
with data sets in which the number of events per variable is much 
less than 10, as it prevents overfitting by shrinking the coefficients 
compared with those by standard Cox regression (Ambler et al, 
2012). The tuning parameter that controls the amount of shrinkage 
of regression coefficients by the ridge penalty was optimised by 
maximising the average of the 10-fold cross-validated partial 
likelihood over 10 different random partitions of the data into 
10-folds (Ambler et al, 2012). We decided to include all preselected 
variables irrespective of significance as nomogram parameters 
as even non-significant variables must be assumed to have a 
non-zero effect on survival. Correlation analysis between clinical 
variables was performed using Spearman and Kendall tau-b rank 



correlation coefficients. Interactions of any predictors with FIGO 
stage were evaluated by comparing the cross -validated partial 
likelihood with and without these interactions. Likewise, we 
checked for non-linear effects of age or LNR by employing the 
technique of fractional polynomials and compared the cross- 
validated partial likelihood with and without nonlinear modelling 
(Royston and Altman, 1994). We used the R package glmnet 
(R Foundation for Statistical Computing) for estimating the model 
coefficients (Simon et al, 2011). 

Model validation 

To determine calibration and discrimination of the model 
corrected for optimism, we generated 1000 bootstrap samples, 
which were used as training sets in a bootstrap validation 
procedure. Specifically, a model was built on each of the 1000 
training sets following the same process as described above. The 
1000 models were then evaluated on the original data (test set) 
without modification, estimating 5-year survival probabilities for 
each patient. For each of the validation models, we constructed a 
time-dependent receiver operating characteristic (ROC) curve 
using the survival ROC package of R (Heagerty et al, 2000). As a 
measure of discrimination, we computed the median and IQR over 
the optimism-corrected time- dependent areas under the ROC 
curves (the c-indices) obtained in each of the models and provide a 
summary c-index as described previously (Steyerberg, 2009 and 
Uno et al, 2011). The calibration of the prediction model was 
assessed by stratifying the patients, by their risk score, into four 
quartiles, and evaluating the observed 3- and 5-year survival 
probability in each quartile by Kaplan-Meier analysis. The mean 
predicted 3- and 5-year survival probabilities, averaged over the 
1000 models, were compared with the averaged observed survival 
probabilities in calibration plots. If the model is well calibrated, 
this plot should demonstrate agreement of the predicted and 
observed survival probabilities. 

RESULTS 
Patients 

A total of 692 consecutive patients with invasive cervical cancer, 
who were treated at the two institutions between January 1999 and 
December 2009, were identified. Of these, 528 patients (Depart- 
ment of Gynecology, Comprehensive Cancer Center Vienna, 
Medical University of Vienna, Vienna, Austria, n = 335 and 
Innsbruck Medical University, Tirol, Austria, n = 193) had all 
nomogram variables available and were included into the analysis. 
Patients' characteristics and treatment modalities are shown in 
Tables 1 and 2, respectively. 

Survival analyses 

Median follow-up time was 45.7 months. Patients' status at last 
follow-up and recurrence status are shown in Table 1. Five-year OS 
rates (standard error) for patients with FIGO stage lA, IB, II, III, 
and IV were 99.0% (1.0), 88.6% (3.0), 65.8% (5.2), 58.7% (11.0), 
and 41.5% (14.7), respectively. Figure 1 shows cervical cancer- 
specific survival with number of patients at risk for cancer-related 
death for each of the follow-up time points. 

Construction and validation of the prediction model 

The FIGO stage, tumour size, age, histologic subtype, LNR, and 
parametrial involvement were predefined as predictors. The FIGO 
stage was strongly correlated with tumour size (tau = 0.82, 
P< 0.001), parametrial involvement (tau = 0.71, P< 0.001), LNR 
(rho = 0.42, P<0.001), and age (rho = 0.33, P<0.001). Univariate 
hazard ratios with 95% confidence intervals estimated from 
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Table 



Patients' characteristics 



Parameter 



N (%) or mean (s.d.) 



Total number of patients enroled 
Age at first diagnosis (years) 

FIGO stage 
FIGO lAI 
FIGO IA2 
FIGO IB I 
FIGO IB2 
FIGO HA 
FIGO IIB 
FIGO IIIA 
FIGO IIIB 
FIGO IVA 
FIGO IVB 

Tumour size 
<2cnn 
^2cm 

Histologic subtype 

Squamous cell carcinoma 

Adenocarcinoma 

Others 

Lymph node status 
Negative 
Positive 

Lymph node ratio^ 
Parametrial involvement 



528 
47.9 (13.5) 



I I I (21.0%) 

I I (2.1%) 
1 69 (32.0%) 
44 (8.3%) 
24 (4.5%) 
I I 8 (22.3%) 

3 (0.6%) 
34 (6.4%) 
7 (1.3%) 
7 (1.3%) 



215 (40.7%) 
3 1 3 (59.3%) 



421 (79.7%) 
86 (16.3%) 
21 (4.0%) 



389 (73.7%) 
1 39 (26.3%) 
0.058 (0. 1 6) 




:g 0.6 



■> 0.4 

CO 



0.2 



0.0 



Patients at risk: 
518 423 334 



275 



208 



156 



87 



49 28 



2 3 4 5 

Overall survival time (years) 



Figure I Kaplan-Meier OS curve for patients with invasive cervical 
cancer (N = 528). 



Table 3 Univariate hazard ratios for overall sunyival for predictors 
that were used in the nomogram 

Univariate hazard ratio 
Variable (95% confidence interval) 



Yes 


170 (32.2%) 


Age (continuous, per decade) 


1.14 (0.97, 1.35) 


Recurrence status 

No. of patients with recurrent disease 

Status at last observation 

Alive or non-cancer related death 
Cancer-related death 


1 1 0 (20.8%) 

451 (85.4%) 
77 (14.6%) 


FIGO stage 
FIGO lA 
FIGO IB 
FIGO II 
FIGO III 
FIGO IV 


1 (ref) 
12.1 (1.6, 90.1) 
36.3 (5.0, 264.9) 
67.6 (8.8, 5 1 8.5) 
1 1 I.I (13.6, 903.8) 


Abbreviations: FIGO = International Federation of Gynecologists and Obstetricians; 
s.d. = standard deviation. ^Lymph node ratio is defined as ratio of positive and totally 
removed lymph nodes. 


Tumour size 
<2cm 
2-4 cm 
>4cm 


1 (ref) 
6.7 (3.0, 14.7) 
8.2 (3.8, 17.5) 


Table 2 Treatment of 528 patients diagnosed with invasive cervical 
cancer 


Histologic subtype 

Squamous cell carcinoma 
Adenocarcinoma 


1 (ref) 
1 .5 (0.89, 2.6) 
1.8 (0.7, 5.1) 


Treatment 


N (%) 


Other histologies 


Surgery 
Conization 
Simple hysterectomy 
Radical hysterectomy or trachelectomy 
Pelvic exenteration 
Pelvic lymphadenectomy 
Paraaortic lymphadenectomy 

Radiotherapy 
No 


72 (13.6%) 
39 (7.4%) 
217 (41.1%) 

2 (0.4%) 
377 (71.4%) 
1 30 (24.6%) 

286 (54.2%) 


Lymph node status 

Negative lymph nodes 
Any positive lymph nodes 

If any positive lymph nodes: 

Lymph node ratio (continuous) 

Parametrial involvement 
No 
Yes 


1 (ref) 
4.8 (3.1, 7.7) 

3.4 (1.3, 9.0) 

1 (ref) 
4.7 (2.9, 7.5) 


Yes 


242 (45.8%) 


Abbreviation: FIGO = International Federation of Gynecologists and Obstetricians. 


Ghemotherapy 
No 
Yes 


353 (76.9%) 
175 (33.1%) 


coefficients by the largest shrinked coefficient obtained in the 
analysis (that of LNR), and multiplying by 100 (Figure 2). 
Including interactions of FIGO stage with the other predictors 
did not improve the cross-validated partial likelihood, and neither 
did the consideration of non-linear effects for LNR or age (data not 
shown). 


univariate Cox regressions are shown in Table 3. From the 
multivariable penaHsed Cox regression model, we computed 
nomogram points for each predictor by dividing the shrinked 
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Total points 0 20 40 60 80 100 120 140 160 180 200 220 240 260 

r 1 1 1 1 T 1 1 i 1 1 1 1 1 

3-year OS rate 96 94 92 89 86 81 75 68 59 49 37 26 16 8 
5-year OS rate 93 90 87 83 77 70 62 52 41 30 19 11 5 2 

Figure 2 Nomogram to predict 3- and 5-year OS using six easily 
available clinical characteristics. To use the nomogram, locate patient's 
variable on the corresponding axis; draw a line to the points axis, sum the 
points, and draw a line from the total points axis to the 3-year OS 
probability axis. 



The prediction model was internally validated using bootstrap 
resampling, assessing its optimism-corrected discrimination 
and calibration. Across the 1000 bootstrap repetitions, the median 
optimism-corrected summary c-index for predicting survival was 
0.723 (25th and 75th percentiles, 0.701 and 0.743, respectively). 
Figure 3 shows a plot of the time-dependent optimism-corrected 
c-index, indicating constant and adequate discriminative ability 
of our prediction model for prediction horizons between 1 and 
5 years. The bootstrap-predicted and the actual OS probabilities at 
3 and 5 years are shown in the calibration plots (Figures 4A and B, 
respectively). The calibration plots describe how far predictions 
are from actual outcomes and how the prediction model fits the 
data. The performance of the model appears to be very accurate, 
within the actual outcomes. 

It is important to note that all six covariates must be available 
to use the nomogram and predict individual patients' OS. For 
example, a 45-year-old (7.2 points) with stage IB (0 points), tumour 
size ^2 cm (0 points), squamous cell carcinoma (0 points) with 
negative lymph nodes (0 points) has a total of 7.2 points, which 
corresponds to the 3- and 5-year OS probabilities of 95% and 
91.8%, respectively. On the other hand, a 45-year-old (7.2 points), 
with stage IIB (20 points), tumour size ^2 cm (0 points), 3 positive 
out of 15 removed nodes (LNR 0.2 corresponding to 50 points), 
squamous cell carcinoma (0 points), and parametrial involvement 
(24 points) has a total of 101.2 points, which corresponds to the 
3- and 5-year OS probabilities of 81.1% and 70.5%, respectively. 

Figure 5 provides predicted 5-year OS probability sub-grouped 
by 2009 FIGO stage. We show that, although FIGO stage is an 
important predictor of 5-year survival probability, the other 
predictors used in our model add further information useful to 
discriminate patients with poor or favourable prognosis. Overall, 
our prediction model combines prognostic factors in a multi- 
variate setting, and thus it is able to discriminate high-, moderate-, 
and low-risk patients in a detailed way through individualised 
predictions. 
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Figure 3 Time-dependent discrimination curves. Optimism-corrected 
area under the ROC (AUROC): median over 1000 bootstrap replicates 
shown as solid line, dashed lines denote 25th and 75th percentiles. 



DISCUSSION 

Accurate prediction of cancer control after definitive treatment for 
cervical cancer is important for patient counselling, follow-up, and 
treatment planning. We constructed a novel nomogram based on a 
model to predict 3- and 5-year OS for patients with invasive 
cervical cancer after surgical staging. The nomogram can be used 
to predict patients' prognosis individually and more accurate than 
FIGO stage alone and is based on the following six easily available 
parameters: FIGO stage, tumour size, histologic type, LNR, age, 
and parametrial involvement. The model was derived from a 
European cohort with the entirety of the age spectrum. 

FIGO stage is the most important prognostic parameter in 
patients diagnosed with cervical cancer. Many, but not all, of the 
known prognostic factors are captured by the FIGO staging 
system. The FIGO staging system is largely based upon physical 
examination and a limited number of diagnostic studies (Pecorelli 
et aly 2009). The limitations of FIGO clinical staging are well 
appreciated and under-staging occurs (Hricak et al, 2005; Quinn 
et aly 2006). Tumour size is an established prognostic parameter 
that is independent of stage and was originally reflected in FIGO 
stages IA-IB2. During the 2009 FIGO staging system changes 
tumour size was added as parameter for stage IIA to distinguish 
between tumour diameters smaller (IIAl) and larger (II A2) than 
4 cm tumours due to its considerable prognostic value (Delgado 
et a/, 1990; Pecorelli et al, 2009). The effect of tumour histology on 
outcome for women with cervical cancer has been actively debated. 
Recent studies have shown that adenocarcinomas are more 
aggressive and are associated with decreased survival for both 
early and advanced- stage carcinomas (Galic et al, 2012). 

Lymph node ratio is a parameter incorporating not only 
information on the number of positive nodes but also the number 
of removed nodes. Lymph node ratio is a useful prognostic 
parameter for patients with cervical cancer and allows stratifica- 
tion of patients into distinctive outcome groups (Polterauer et al, 
2010). It is important to highlight that assigning LNR requires 
surgical staging with node dissection. The authors are aware that 
the therapeutic value of pretreatment surgical lymph node staging 
for patients with advanced stage cervical cancer is still contro- 
versial (Brockbank et al, 2011). At our institution extended- field 
radiotherapy is recommended when paraaortic nodal disease is 
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Figure 4 Cervical cancer OS nomogram calibration plots: (A) 3-year and (B) 5-year nomogram calibration curve. Nomogram-predicted OS rates are 
plotted on the x-axis, actual OS are plotted on the /-axis. The dashed line represents the ideal fit where the nomogram-predicted probability matches the 
observed probability from Kaplan-Meier estimates. 
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Figure 5 Predicted 5-year OS probability by 2009 FIGO stage. 



proven and we observed excellent pelvic control in these high-risk 
patients using this concept (Potter et al, 2011). For patients with 
stage lAl disease lymph node, dissection was only performed when 
LVSI was present. As previous studies have shown that the risk 
for nodal involvement is less than 1% for patients with stage lAl 
disease and without LVSI, we assumed negative nodal status and 
this group of patients was allocated a LNR of 0.0. 

Recent studies suggested that younger age is an unfavourable 
prognostic factor, especially in more advanced stages (Chen et al, 
1999). Survival analysis revealed that younger patients showed 
impaired survival in cohort and therefore age was included into 
our model. Parametrial involvement significantly influences out- 
come of patients with cervical cancer but is not reflected in FIGO 
stages III-IV (Takeda et al, 2002). In the Gynecologic Oncology 
Group (GOG), 49 study patients with apparent stage I disease with 
and without parametrial involvement were shown to have 70% and 
85% 3 -years disease-free intervals, respectively (Delgado et al, 
1990). Patients found to have parametrial involvement will usually 
receive primary or adjuvant radiation therapy after surgery 
(Committee on Practice Bulletins- Gynaecology, 2002). 

A nomogram is a graphic prediction tool that incorporates 
clinical risk factors already included in established staging 
systems, as well as other additional, clinical, and pathologic 
factors known to have an impact on outcome. A distinct advantage 
of a nomogram is that all the critical variables that determine 



outcome can be graphically displayed (lasonos et al, 2008). 
Prognostic nomograms attempt to combine important clinical 
factors to quantify the risk as precisely as possible to accurately 
predict clinical outcome. Clinical nomograms have been developed 
as predictive tools for outcomes in gynaecologic malignancies such 
as endometrial and ovarian cancer (Chi et al, 2008; Abu-Rustum 
et al, 2010). For the estimation of outcome of patients with cervical 
cancer, few nomograms have been published so far (Kim et al, 
2010; Tseng et al, 2010). This nomogram is the first to predict OS 
through stages I-IV that was constructed based on data of a mainly 
Caucasian patient cohort. 

Despite having achieved prognostic accuracy, our study is not 
devoid of limitations. The multi-institutional nature of our data 
set may be interpreted as a limitation, given that it groups the 
contribution of multiple surgeons and pathologists and relies on 
different surgical approaches, in addition to other differences that 
might distinguish the two contributing centres. However, this 
limitation could also be seen as strength, as it makes our 
conclusions more general and increases the available sample size, 
which is important given the low event rate. Despite combining 
data from two institutions, we observed a limited number of events 
(i.e., cancer-related deaths). To take this into account, the model 
coefficients were computed using the recently developed 'ridge' 
method. This approach was originally designed to deal with 
situations where the number of predictors exceeds the number of 
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events by large CP» n') such as in genomic applications, and was 
recently found to be superior to other methods also for prediction 
in 'classical' ('P<n') settings, where less than 10 events per variable 
are available (Ambler et al, 2012). In ridge regression, the amount 
of penalisation is driven by predictive performance in cross- 
validation (cross-validated partial likelihood). This 'automatism' 
guards against over-fitting a model. However, shortage of data, 
caused by too many variables compared with the number 
of events, may lead to higher penalisation, and reduce ability of 
a model to predict the outcome in the extremes. Therefore, we 
created calibration plots based on internal cross-validation by 
bootstrap resampling, which showed an almost perfect agreement 
of observed and predicted event rates. Nevertheless, external 
validity of our model is a crucial prerequisite to clinical 
applicability, and can only be assessed by confirming results in a 
reasonably large independent validation cohort with an adequate 
follow-up period. Another potential limitation of our model as 
typical for nomograms is that construction is based on retro- 
spective data. Therefore, our study might be limited by biases such 
as lack of random assignment, patient selection, and incomplete 
data acquisition. Nomogram variables were missing in a significant 
number (n = 164) of patients of our cohort. Mostly information on 
parametrial involvement and exact tumour size was incompletely 
documented in the cases that were excluded from analysis, 
especially in patients with advanced stage cervical cancer. 



In summary, we present the first model to predict 3- and 5 -year 
survival for patients with cervical cancer after surgical staging that 
is applicable through stages I-IV. The model was developed in 
a relatively large European, mainly Caucasian, cohort and can be 
applied for prediction by means of the presented nomogram or, 
more conveniently, by an online prediction tool (available at 
http://www.ccc.ac.at/gcu). 
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